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Reported is a study to investigate probleas 
handicapped readers had with the Interaediate Science Curriculua 
Study (ISCS) test and of a aethod for adapting the test so that it 
might be aore understandable. Seven questions relatod to the 
1968-1969 ISCS test Probing the Natural World, Voluae I nere studied. 
The investigations of these questions vas undertaken in tvo stagest 

(1) a pilot study using six classes in one junior high school, and 

(2) the aajor study vhich involved the testing of 28 seventh-grade 
ISCS classes at five junior high schools. The pilot study findings 
indicated the need for a revised test. In addition, achieveaent over 
the school year vas exaained using a repeated aeasure design. Itea 
analysis data vere used to see vhat changes in the psychoaetrics of 
the two instruments occurred as a result of revision. The Hotelling 
t-test was used to see if the revised ISCS test had a reduced 
correlation with reading as coapared with the '1968-1969 ISCS test. 
Reading coaprehension was found to be an iaportant factor related to 
achieveaent on both the original and revised tests. The findings 
indicated that the oral-deaonstration technique is a valid aethod to 
coapensate for lack of reading ability. (Author/EB) 
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Introduction 



This paper is an attempt to show how a written 
science test may be adapted so that it is more understandable 
for handicapped readers. 

Problems 

This study involved the following problems: 

1 . Is the 1968-1969 ISCS test Probing the Natural 
World , Volume I, an appropriate instrument for 
measuring science achievement in the case of 
seventh grade pupils two or more grade levels 
behind in reading comprehension in the Philadel- 
phia Public Schools? 

2. Does oral reading of the 1968-1969 ISCS test by 
the test administrator help handicapped readers 
overcome their difficulty in understanding the 
test items? 

3. Does redesigning the 1968-1969 ISCS test produce 
an instrument which indicates that an oral- 
demonstration presentation helps handicapped 
readers to understand those items better than 
ones they read silently? 
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4. Does redesigning the 1968-1969 ISCS test produce 
an instrument which is less dependent on reading 
comprehension as measured by the 1 969 Iowa Tests 
of Basic Skills? 

5. ^'How do test performances on the original ISCS 

test and the Revised ISCS Test compare for the 
handicapped (H) and nonhandicapped (Nil) readers 
taking the ISCS course in the Philadelphia Pub- 
lic Schools during the 1969-1970 school year? 

6. Were the handicapped readers (H) and nonhandi- 
capped (NlO readers* means for the experimental 
oral -demonstration subtest greater than the means 
for the control silent subtest of the Revised 
ISCS Test? 

7. Does redesigning the 1968-1969 ISCS test produce 
an instrument which appropriately measures science 
achievement in the ISCS course for seventh grade 
Philadelphia Public School pupils? 

Procedures 

The investigations of these seven questions was under- 
taken in two stages: (Da pilot study using six classes in 
one junior high school, and (2) the major study which in- 
volved the testing of 28 seventh grade ISCS classes at five 
junior high schools, Teachers whose classes were used in 



the study participated in the ISCS ia-servicc workshop at 
TcnipLe University nrxcl volunteered in March 1970 to have their 
classes »esLed in June using the Revised ISCS Test, Test 
data on the samct sample using the original ISCS test was made 
available from a companion study by Clark (1973) to answer 
questions 1, ^, and 5, 

The pilot study investigated problems 1, 2, and 3 to 
find out if a problem with readability of the original ISCS 
test was present- A junior high school with two ISCS teachers, 
each having three seventh grade ISCS cla^ses^"" was used for the 
pilot study. In order to answer question 1 mean reading com- 
prehension levels were computed for each class. Reading com- 
prehension scores were obtained from the 1969 administration 
of the Iowa Tests of Basic Skills (ITBS), Classes which had 
mean reading comprehension two or more years below the readabil 
ity level of the original ISCS test of seventh grade third 
month as calculated usinj^ the Dale-Chatl Formula were categor- 
ized as having difficulty reading the original ISCS test. 

P'or classes which were categorized as handicapped in 
their ability to read the original ISCS test, a Jt-tcst was 
calculated to detenriine if H classes did better on the March 
ISCS posttest than the chance scores of 11 for a item 
multiple choice test, This March ISCS posttest was admin- 
istered by the teachers as part of Clark* s companion 
study. 

In order to detennine whether the mere oral reading 
of the original ISCS test might help handicapped readers 



tho following design was implemented to answer question 2. 

Tho pupils in the six ISCS classes wore matched in ^ 
one oL^ two way«: (1) on sell using a split-halves treatment 
of the original ISCS test, or (2) with a pupil of the same 
sex with similar academic achievement ♦ The experimental 
treatment consisted of an oral presentation of some of the 
items on the 1968-1969 version of the ISCS test. In the 
control presentation pupils read items silently as they had 
in the March ISCS posttest. It was hypothesized that the 
oral reading might lead to higher scores on the oral subtest 
as compared to the silent subtest, particularly for H read- 
ing classes, Nonparametric tests were used to test for the 
significance of the difference between the experimental and 
control groups. Failure to find significant differences 
favoring the oral treatment for H classes would be inter- 
preted as evidence that the mere oral reading of the original 
ISCS test was not sufficient to help increase pupils* under- 
standing of the items, 

A revision of the 1968-1969 ISCS test was undertaken 
to make some of the items more understandable* Additional 
pictures were added and vocabulary and sentences were changed 
in an attempt to increase the readability. Half of the 38 
items selected from the original ISCS test were presented 
orally with demonstrations using the ISCS science equipment 
available in the classroom. A t^-test of the difference was 
calculated to find out if pupils performed better on the 
orally-demonstrated subtest as compared to the silent 



subtest. If the oral -demons trntion subtest mean was found 
to bo sij\ni r iiN'^mi 1 y abov(^ t lio silent mean, the major «tudy 
Wduld I iivoHl i j»,nL<» (jUo«li<)U« I, ^,5, (> , and 7 . 

In the major study pupils from five junior high 
schools were administered the Revised ISGS Test in June 
1 970 . Pupils present for the September, March, and June 
administrations of the ISGS tests who had 1969 ITBS reading 
comprehension subscores were used in the analysis of the 
data. There were 188 handicapped readers (H group) and 
2^2 nonhandicapped readers (NH group) whose data were used 
to answer the questions. 

Class means for the 1970 ITBS reading comprehension 
subtest were computed to categorize classes on their ability 
to read the original ISGS test. If the reading comprehen- 
sion grade equivalent mean score for a class was two years 
or more below the readability level of the original ISGS 
test (i.e., seventh grade third month), then that class was 
categorized as handicapped in reading the ISGS test and was 
placed in the il group. All classes above the cut-off point 
of fifth grade third month were considered to be nonhandi- 
capped in their ability to read the original ISGS test. 
These classes are referred to as the NH group. 

To answer question 1 to see if the conclusion in the 
main study replicated the pilot study findings the fiducial 
limits of the original ISGS test were computed for the H 
group. The investigator wanted to know whether the H group 
means on the ISCS test were significantly different from 



the chance means of the September and March administrations 
of the original ISCS test. Failure to find a positive sig- 
nificant difference in the March posttest v/ould be consid- 
ered as evidence of the inappropriateness of the 1968-1969 
version of the ISCS test for handicapped readers. 

Correlations were computed between the two ISCS tests 
and the 1969 reading comprehension ITBS score to answer 
question 4. Since both the original ISCS test and the Re- 
vised ISCS Test required some reading it was of interest to 
see if the correlations with reading were different for the 
two forms of the test. An analysis of H group data was per- 
formed to see if a difference between the March original 
ISCS test correlation with reading and the Revised ISCS Test 
correlation with reading was significant using a Hotelling 
Jt-test. In addition a comparative study of H and NH groups 
was undertaken to see if science achievement differed sig- 
nificantly between the different reading ability groups. 

Comparisons of the H and NH groups* achievement as 
measured by the original and the Revised ISCS tests were 
made using analysis of variance to compute an F ratio. Is 
significant difference in achievement favoring the NH group 
(i.e., able reading classes) would be interpreted as evidenc 
that reading comprehension ability was an important factor 
which may have limited the amount of science learning that 
was measurable using the ISCS tests. The results from the 
analysis of H and NH groups' science achievement would be 
used to answer question 5. 



To answer question 6 the Revised ISCS Test was de- 
signed with two subtests: (t) an oral -demonstration sub- 
test, and (2) a silent subtest. The oral-demonstrat ion items 
were alternated with silent items to determine if there was 
a treatment effect resulting from the oral -demonstration pres 
entation. The oral set and silent set of items, which were 
taken from the 1968-1969 form of the ISCS test, were of 
like difficulty according to March post test analysis (i,e,, 
D *41 ) for the NH group, but had a slight difference in 
difficulties of .05 for the H group, with the oral set being 
slightly easier. Thus in the case of the H group there may 
have been a slight difference in difficulties between the 
two subtests before the items were revised and the experi- 
mental oral-demonstration administration was applied. It 
was presumed that revisions of both the oral and silent 
items would lead to changes in difficulty. However, in 
addition to the written revisions, the oral -demonstration 
presentation may have made certain items less difficult. 

A Jt-test of the difference between the oral-demon- 
stration subtest mean and the silent subtest mean was used 
to see if the treatment aided H and NH pupils in understand- 
ing the items better, enabling them to achieve higher scores 
on the oral subtest than the silent subtest. If the Revised 
ISCS Test means for the H and NH groups were above the 
chance level mean of the test and if the experimental oral 
treatment mean was significantly greater than the nontreat- 
ment silent subtest mean, then there was some evidence that 
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the oraL-clcmonstratlon technique, contributed to the design- 
ing of an appropriate instrument, 

A questionnaire was given to 11 and Nil classes to com- 
pare pupils^ I'eelings toward the original and Revised ISGS 
tests. The results of the questionnaire were utilized to 
answer question 7 in conjunction with H group test results. 

In addition a repeated measures design was used to 
see if the H group made significant gains in science in 
order to answer question 7. September, March, and June ad- 
ministrations of ISGS tests were used to measure science 
achievement during the 1969-1970 school year. If the over- 
all F ratio was found to be significant, then the mean dif- 
ferences betv^reen September and March, and March and June 
tests would be analyzed using a Tukey test. Failure to show 
significant gains in H group pre- to-post test means would be 
interpreted as due to the inappropriateness of one or both 
instruments. 

Pilot Study Findings 

Q uestion 1 . Is the 1968-1969 ISGS test Probing the 
Natural Worl d, Volume I, an appropriate instrument for 
measuring science achievement in the case of seventh grade 
pupils two or more grade levels behind in reading compre- 
hension in the Philadelphia Public Schools? 

No, the 1968-1969 version of the ISGS test was in- 
appropriate for measuring the science learned by H classes. 
The March ISGS posttest means for the two handicapped 
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classes wore found to bo within the chance cti strlbutlba of 
test scores (i.e., see Table 3 in Appendix). 

Quest ion 2 > Does oral, rvadlng <>1 the 1968-1969 ISGS. 
test by the test administrator help handicapped readers 
overcome their difficulty in understanding the test items? 

No, handicapped reading classes were not aided by 
orally presenting parts of the original ISCS test. As indi- 
cated on Table 4 in the Appenclix, results from nonparametric 
tests of the difference between oral and silent treatments 
revealed that the H classes were not helped by the test 
administrators' oral presentation of certain items. One H 
class did significantly better on the silent subtest. 

Q uestion 3 . Does redesigning the 1968-1969 ISCS 
test produce an instrument which indicates chat an oral- 
demonstration presentation helps handicapped readers to 
understand those items better than ones they read silently? 

Yes, the oral -demons tration subtest mean scores were 
significantly above the silent subtest mean for the H classes 
as indicated on Table 6 in the Appendix. 

Major Study Findings 

Question 1 was asked again to see if the finding of 
the pilot study was replicated in the major study. 

Question 1 . Is the 1968-1969 ISCS test P robing the 
Natural World s Volume I, an appropriate instrument for 

ERLC 



10 

measuring science achievement in the case of pupils two or 
more grade levels behind in reading comprehension in the 
Phi 1 ndol phia Publ i<' Sc^hool ss' 

No, the 1 %8-1 969 version of the ISCS test was in- 
appropriate for measuring the science learned by the H 
group. This finding was in agreement with the pilot study 
conclusion. As indicated on Table 13 in the Appendix, the 
ISCS mean for the H group on the March posttest using the 
original ISCS test was not statistically significant in its 
difference from the chance score of 11 for the kk item test. 

A gain score analysis of September and March ISCS 
test results are presented in Table 16. A Tukey test of the 
difference between March 1970 and September 1969 means using 
the original ISCS test was not significants This further 
supports the conclusion that the original ISCS test was not 
appropriate for measuring progress of H classes taking the 
ISCS course. 

Question 4 . Doef? redesigning the 1968-1969 ISCS 
tost produce an instrument which is less dependent on reading 
comprehension as measured by the 1969 Iowa Tests of Basic 
Skills? 

No, the Revised ISCS Test was found to be signifi- 
cantly more correlated with reading comprehension than the 
original ISCS test. The results of the Hotelling t-test are 
in Table 23. The correlation of the 1969 ITBS reading com- 
prehension grade equivalent scores increased significantly 
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from the administration of the original ISCS test in Marcli 
nnd tVic Revised ISCS Test administered in June. This find- 
ing was a surprise; however, it was not knowi when the study 
was undertaken that the correlation bctt^een the 1969 ITBS 
reading comprehension subtest and the 1968-1969 fom of the 
ISCS test was near zero. Restriction of the range of pos- 
sible ITBS scores as a result of selecting classes in the 
H group on the basis of their being t-wo years below the 
readability level of. the ISCS test may have been part of 
the reason for the neai" zero correlation. In addition, 
random test taking behavior on either the March ISCS test 
or the 1969 ITBS test might have contributed to the low cor- 
relation between a science and a reading comprehension test. 

Question 5 . How do test performances on the original 

ISCS test and the Revised ISCS Test compare for the handi- 

capped (i.e., H) and nonha'ndicapped (i.e., NH) readers taking 

the ISCS course in the Philadelphia Public Schools during 

the 1969-1970 school year? 

There was a significant difference between H and 

NH performance on the September 1969> and the March and 
June 1 970 administrations of ISCS tests as indicated in 
Tables 17 and 18, and Figure 2. The significant difference 
in performance be tv</een the H and NU groups on three adminis- 
trations using two versions of the ISCS test indicated that 
reading comprehension achievement levels wore ah important 
factor affecting pupil ability to both learn and show what 
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they had Ic^arncd from thcix' ISCS course. 

Question 6 / Were the handicapped readers (H) and 
nonhandicapped (NH) r^eaders^ mcaj\s for the experimental oral 
demonstration subtest greater than the means for the control 
silent subtest of the Revised ISCS Test? 

Yes, there was a treatment effect favoring the oral- 
demonstration technique over the silent treatment as indi-* 
cated in Table 20 , The mean for the oral-demonstration sub- 
test was significantly greater than the mean for the silent 
subtest for both H and groups. This occurred in spite of 
the fact that many of the items were paired , so that if the 
oral item came f irst j pupils may have been helped in their 
understanding of the next silent item; This treatment effect 
aided in making orally-demonstrated items more understandable 
and may have had a spin-off effect of making some of the 
related silent items more understandable. 

Question 7 > Does redesigning the 1968-1969 ISCS 
test produce an instrument which appropriately measures 
science achievement in the ISCS course for seventh grade 
Philadelphia Public School pupils? 

Yes, pupil performance above the chance level of the 
Revised ISCS Test administered in June indicates that the 
redesigned test was appropriate, The fiducial limits on the 
test performance data for the H group subtests are presented 
in Table 21 . , 

In addition a gain score dnalj^gis ^using the Tukey 
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test revealed that sigaificant gaint; had been made between 
SopLombor and Jimo and March and June, but not between 
.St*|>loinbor auit M/mdi, 'hius, the use of the Riwtwcd 1S0.S 'rcHt 
in June was an Important factor in picking up significant 
gains indicative that some science Learning did occuri The 
results of gain score analysis are presented in Table 1 6 . 

Pupils* responses to a questionnaire also revealed 
that they were able to understand the items better on the 
Revised ISCS Test as compared to the original ISOS test v 

Summary 

The dissertation has investigated the problem of th6 
appropriateness of the ISGS test and adapted the 1968-1969 
version of the ISCS test so that it might be better under- 
stood by handicapped readers who took the seventh grade ISCS 
course in the Philadelphia Public Schools during the 1969- 
1970 school year. Reading comprehension was found to be an 
important factor related to achievement on both the original 
and Revised ISCS tests. The findings in this dissertation 
indicate that the oral-demonstration technique for present- 
ing written items is a valid method to insure that the lack 
of reading ability do.es not block the measurement of science 
learned; Learning to read science materials needs to be a 
major objective of all science teachers if written science 
tests are to be used as the criterion for science learning. 

Some significant, although possibly low, correlation 
should ext^it bG^t't^een a written science test (some of wliicft 
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may bo presented orally) and a reading couiurehension test in 
order to obtain a meaningful measure of science learning. 

The fact that N}I classes also improved significantly 
on the Revised ISCS Test was interpreted as evidence that 
some of the pupils in those classes were also having diffi-' 
culty in reading the original ISCS test. Thus, the investi- 
gator concluded that the special oral-demonstration would 
be appropriate for pupils v/hose reading comprehension level 
was one year below the readability level of the test. 

The rewording of items and addition of pictorial 
representations to 11 of them, plus the oral-^demonstratioh 
presentation, indicated to pupils that the test administrator 
really cared about the science they had learned. Thus, 
the outcome was a response pattern other than the random 
filling-in of tlie answer sheet. More humane techniques 
for administering written tests to handicapped readers 
need to be utilized if valid measures oi learning are 
desired. 
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TABLE 3 

MARCH ISCS MEANS FOR HANDICAPPED READING CLASSES 

AND t TEST 



Statistic 


Class 1 


MARCH ISCS TEST 
Class *4 uiass 




Number of Students 


24 


22 




Test means 


13 • 3 


11 .3 


12.3 


Standard deviation 




3.19 


3.60 


Chance level 
(4U items) 


11 


11 


11 


Difference from 
chance 




.3 


l.^f 


t^ value (d.f.) 




(23) .397 (21 ) 


2.68 <45) 


Probability 


s. 


N.S. 


N.S. 


Level of rejection^ 


.01 


.01 


.01 



^Thc .01 level was chosen for to reduce the risk of a 
Type I error. 1 



TABLE U • 

MARCH IS CS SUBTEST MEANS AND TEST FOR THE SIGNIFICANCE 
OF THE DIFFERENCE BETWEEN ORAL AND SILENT SUBTESTS 



Class Numbers of Class Means on ISCS Subtests Test of 
Match- Items Ad- Significance 
ing ministered N Oral Silent Cp<.05) 



1 


P% • 


23 


7 


8.09 


9,36 


-S. 


2 




36 


25 


7.64 


6.11 


+S. 


3 


P NH 


23 • 


9, 


■ 12.11 


10.11 


• N.S, 


k 


P H 


23 




8.50 


8.16 


N.S. 


5 


s HH 


36 


25 


7.0k 


6.52 


N.S. 


6 


s NH 


36 


'•2.6 


9,H] 


12.26 • 


N.S. 



^Matched ^airs of pupils fWoiCAPPED in reading 

^Pupils, received a split half treatment, and therefore are 
matched on s^el^. fJoNHANDICAPPED in reading ^ 
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TABLE 6 

READING HANDICAPPED GROUP 
t TEST 01? THE DIFFEIUINCE BETWEEN ORAL AND SILENT SUBTESTS 

OK THE REVISED ISCS TEST 



Mean Difference = .98 

Standard Deviation of the Mean = 2.62 

Standard Error of the Mean =. 39 

t Value =2.51 

d.f . ^ if5 

p < .05 
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TABLE 7 

REVISED ISCS TEST MEANS FOR PILOT STUDY CLASSES 



Statistic 


1 




3 4 
Class 


5 


6 


Overall 


Number of Pupils 


2i4 


27 


23 


20 


25 


25 


143 


Test Means 


17.0 


17,5 


17.6 


11 .2 




21 .5 


16.6 


Oral Mean 


8.6 


7 .7 


9.2 






12.1- 


8.8 


Silent Mean 


8.4 


9.8 


8i4 


4.6 


7.5 




ilifflii 


i^pffiiipfWiiP 


.2 


-2.1 


. .8 






iiil 





., - - . . ^:stm. 



TAULH 1 3 

TICST OF Kl DUG J. AT. I/I M ITS OV T.SCS MKANS 
I'OR llANDIC/\inM:i) (ID* RHADICUS 



Admi n i. s t r a t i. on 



C n 1- *i <^ 1* ^ V\ 


September 


March 


June 


N 


188 


188 


188 


M • 


10.10 


11 .35 


13.6 


Chance Score 


11 


11 


9.5 


Z 




A .67 


13.6 


P 




N.S. 





"''p < .01 

^ May be Indicative that the original ISCS test for H group was a 
speeded test rather than a mastery test. . 



TAm.K 16 

ANAIA'.SKS OF MAN 1)1 CAP) Ml I) KHADTNr. Cl.ASfJIiS 
GAIN SC0Ri:8 OVKR T)il<i:i: TIMF. riUUOIJS 



Time Periods 


2-1 


3-1^ 


3-2 


Mean Difference 


1 .25 


3.50 


2.25 


g Values - Critical g^Q =.3.53 


2.^1 


k.25 


3.70 


Level of significance 


N.R. 







p < .05 



This gain score analysis Is based on the assumption that the two 
Torms are para I led . 
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TABLE 17 

A COMPARISON OF ISCS TEST MEANS FOR H.WOLCAPPED (If) 
AND NONHANDICAPPED (NH) READERS 



Administration 


Blocking 




Statistic 




IT- 


" "H" 


snr: 


September 


H 
NH 


188 
242 


10.0 
11.8 


3.52 
3.67 


March 


H 
NH 


188 
242 


11 .3 
16.4 


3.86 
3.91 


June 


H 
NH 


188 
242 


13.6 
19.4 


3.81 
4.76 



Means 




chance 
level for 
original 
ISCS test 



Time 



^A quadratic component was found to be present for the H f^roub 
Linearity of regression was tested and f.ound to be sLnifi-^' 
cant for both the m{ and H groups at the .01 level . 

FIGURE 2 

A COMPARISON OP HANDICAPPED AND NONHANDKIAPPED 

READERS' ISGS ACHIEVEMENT^ • ^ ^ ^ - 



TA13LK 1« 

RESULTS OF ANALYSIS OF VARIANCE 
ON PRE- POST- AND REVISED ISCS MEAN SCORES OF H AND Nil GROUPS 

TO TEST NULL HYPOTHESIS U 

.Source of Variation SS df Ms" p p 

Within Cells 72.68 26 2.79 

Betv;o.en Groupjngs ]6S.5h ) 165.54 59.2 .001 
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TA»LE 20 

TEST OP SIGNIPICANCE 0? THE DIl-FERENCE BETUTCEN WiANS ON 
TWO SUBTESTS OV THE REVISED ISCS TESTS USIKG A 
CORRELATED t TIDST 



Non 

Vorlnblos Handicapped S,D. Handicopped S.D. 



Mean for Oral Tost 



7.33 a.S7 10.11 2.79 

Mean tor Silent Test 1:26 " 2,17 Jt^ 2.77 

Difference 1 '07 '71 

Number of Pupils 188 ' 

.88 .^5 



Pearson r 
t value 



4.3-^" . 3.6 



TABLE 21 

TESTOF FIDUCIAL LIMITS OF THE REVISED ISCS SUBTESTS 

FOR THE H GROUP 



Statistic 



Number of Pupils 
Mean for the Subtest 
Chance Score 
2 Value 

Probability Level 



Subtest 



Oral 



188 

7.^33- 

14'.- §0 
4i« 



Silent 



188 
6.20 



TABLE 23 

TEST OP SIGNIFICANCE BETWEEN TWO CORRELATION COEFFICIENTS 



'^^r -,29r -.17 

ISCS, Iowa Reading Revised ISCS, IOWA Reading 

^ .28 

ISCS, Revised ISCS 

Number of pupils taking all three ISCS tests =188 
d.f. =185 
Vr " -2.01* 

p < .05 
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